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A complete analysis of the canonical structure for a gauge fixed PST bosonic five 
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brane action is performed. This canonical formulation is quadratic in the depen- 
dence on the antisymmetric field and it has second class constraints. We remove 
the second class constraints and a master canonical action with only first class 
constraints is proposed. The nilpotent BRST charge and its BRST invariant effec- 



tive theory is constructed. The construction does not assume the existence of the 
inverse of the induced metric. Singular configurations are then physical ones. We 



obtain the physical Hamiltonian of the theory and analyze its stability properties. 
Finally, by studying the algebra of diffeomorphisms we find under mild assump- 
tions the general structure for the Hamiltonian constraint for theories invariant 
under 6 dimensional diffeomorphisms and we give an algebraic characterization of 
the constraint associated with the bosonic five brane action. We also identify the 
constraint for the bosonic five brane action upgraded with a cosmological term, it 
contains a Born-Infeld type term. 



1 E-mail address: alex@fis.usb.ve 
2 E-mail address: arestu@usb.ve 



1 Introduction 



In recent years the M5-brane in D = 11 has acquired a protagonist role in understanding the 
duality relations in the M-theory framework. The degrees of freedom of 6d world volume theory 
of the M5-brane are associated with the super-symmetric tensor multiplet N = (2,0). This 
multiplet contains a 2- form gauge field Bmn with a self-dual field strength, five scalars and two 
chiral spinors. The presence of this chiral gauge field was an obstacle to the formulation of a 
covariant world volume M5-brane action. P. Pasti, D. Sorokin and M.Tonin in 1997 |I[ found 
a way of dealing with this obstacle using the auxiliary scalar field approach, thus allowing 
the formulation of a covariant super-5-brane action with its usual K-invariance 0. However, 
in order to obtain an operatorial or functional integral quantum formulation of the theory, 
for example to analyze the quantum stability properties of the theory, it seems necessary to 
eliminate by partial gauge fixing the auxiliary scalar field of the PST approach, since it is 
present in the denominator of the Lagrangian. The covariant field equations for the super 
M5-brane were first obtained in |§, f§] and || using the superembedding approach. 
An action for the M5-brane was independently formulated in a non manifestly covariant way 
in ||, it corresponds to a gauge fixed version of the covariant proposal. There are two par- 
tial gauge fixing conditions that naturally eliminate off the auxiliary scalar field of the PST 
approach. One of them corresponds to fixing the scalar field as the world volume time and 
the other as a local world volume spatial coordinate. The latest corresponds to the formulation 
in ||. In the first case the action is first-order in time derivatives of the antisymmetric field 
and in the second one is higher order in time derivatives, moreover it is non polinomial in 
the dependence of the time derivatives of the antisymmetric field. The first case, although 
has second class constraints, allows a direct canonical analysis. We remark that the PST 
formulation as well as the formulation in || assume the existence of the inverse of the induced 
metric. 

The canonical Hamiltonian for the 5-brane was first obtained in 0, || and also discussed in 
0, however in order to analyze the stability properties of the 5-brane and to compare them 
with the D — 11 supermembrane theory it is relevant to go one step further, to include also 
in the analysis the configurations with zero determinant of the induced metric since as we will 
show they are physical configurations of the theory, and study then the physical Hamiltonian. 
That is, the Hamiltonian describing the dynamics of the physical degrees of freedom once the 
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gauge ones have been eliminated by an admissible gauge fixing procedure. 

When the theory is reduced to the subspace of solutions of the field equations for which the 

induced metric is flat, the Lagrangian of the 5-brane reduces to the one in jnj whose canonical 



analysis was obtained in ||11|| . An interesting feature of [TlJ was that the second class con- 
straints of the original formulation were eliminated giving rise to a canonical Lagrangian with 
first class constraints only In this case, its double dimensional reduction yield a canonical 
formulation allowing two covariant gauge fixing procedures, one of them gives the Hamiltonian 
formulation of the 4-brane and the other one its dual in terms of the antisymmetric field. The 
analysis was performed even though the dependence on the time derivatives of the antisym- 
metric field is non polinomial. The resulting Hamiltonians contain in both cases the typical 
Born-Infeld structure for the field strength of the gauge vector field and antisymmetric one. 
In this work we give first a complete analysis of the canonical structure of the bosonic sector 
of the 5-brane. We start from the PST action and consider the partial gauge fixing where 
the scalar field is fixed as the world volume time. The canonical formulation of the M5-brane 
turns out to be quadratic in the dependence on the antisymmetric field, and has second class 
constraints together with the first class ones which generates the symmetries of the theory. The 
Hamiltonian in || reduces to the one we obtain once their constraints are properly combined. 
We remove the second class constraints preserving the locality of the field theory and ending 
up with a master canonical action which only contains first class constraints and is well defined 
even for singular induced metrics. We refer as singular configurations the ones for which the 
determinant of the spatial part of the induced metric is zero at some point or neighborhood 
of the world volume. The algebra of the 6 dimensional diffeomorphisms generated by the first 
class constraints is explicitly given. It is an open algebra. 

This is the first step in the construction of an unconstrained extended phase where a realization 
of the nilpotent BRST charge may be possible. The BRST charge is a fundamental geometrical 
object in the formulation of the quantum field theory associated to the canonical Lagrangian. 
It may also be an important geometrical object in the construction of the Seiberg-Witten 
map [0] that relates the 5-brane theory formulated in terms of a commutative geometry 
to a formulation in terms of an associated noncommutative one. The Seiberg-Witten map 
is a one to one correspondence between gauge equivalent classes, which correspond to BRST 
equivalence classes in the extended phase space. In particular, this implies that the cohomology 
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classes of the BRST operators in the commutative realization and in the non commutative 
one are in one to one correspondence. We construct the nilpotent BRST charge of the theory 
and its BRST invariant effective theory. The construction involves several steps beyond the 
standard construction for a closed algebra. In fact, one has to introduce the higher order 
structure functions of the open algebra. We obtain its physical Hamiltonian and analyze its 
stability properties remarking that although the Hamiltonian is quadratic on the dependence 
on the antisymmetric field, its reduction to a flat induced metric gives the Born-Infeld type of 
Hamiltonian. The improvement is important when the operatorial formulation of the BRST 
invariant effective action is considered. 

Finally, by studying the algebra of diffeomorphisms we find the most general structure for 
the Hamiltonian constraint in terms of the membrane maps and the antisymmetric field. 
The corresponding canonical Hamiltonian describe field theories which are invariant under 
diffeomorphisms over a 6 dimensional world volume with a chiral gauge field. The unique 
Hamiltonian constraint with quadratic dependence on the antisymmetric field corresponds to 
the M5-brane. We also identify the constraint associated to the bosonic M5-brane action 
upgraded with a cosmological term,it involves a Born-Infeld type of term. From the algebraic 
point of view, the Hamiltonian constraint for the M5-brane is characterized by being the 
unique one which is polinomial in its dependence on the field strength of the antisymmetric 
field and is well defined even for singular induced metrics. The BRST invariant action, that 
we will construct, is characterized also by the same property. Its Lagrangian density is well 
behaved even for singular configurations of the induced metric. Hence, singular configurations 
of the metric are allowed as physical configurations. In distinction, the PST covariant action, 
as well as the one in , assume the existence of the inverse of the induced metric at any point 
of the world volume. In the case of the D=ll Supermembrane, the singular configurations are 
also physical configurations. This has important consequences with respect to the quantum 
properties of the theory. In fact, the existence of these configurations together with the 
supersymmetry render the spectrum continuous from zero to infinite. They are also related 
to the topology changes on the embedded surface in the target space, without changing the 
energy of the supermembrane. It is thus very important to take them into account in any 
analysis of the theory. 

Another consequence of the existence of singular configurations is that the static "gauge" 
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is not a correct gauge fixing of the theory, since giving a singular configuration one cannot 
gauge transform it to a non singular one (in the static gauge the spatial part of the induced 
metric is always non singular). If one restricts the theory to the space of nontrivial higher 
order bundles JT3J or equivalent to non trivial wrapping of the 5-brane on the target space, 
then the singular configurations are avoided and it becomes a correct gauge fixing condition. 
However the quantum stability analysis of the theory must include all canonical admissible 
configurations and it is in that case that the static gauge is not allowed. In distinction the 
light cone gauge is always an admissible gauge. In fact, even the singular configurations can 
be gauged to it, and of course they remain to be singular. 

2 Canonical analysis of the bosonic 5-brane action 

We start from the PST action for the M5-brane and consider the gauge in which the scalar 
field is proportional to the world volume time. This gauge fixing, associated to the gauge 
symmetry of the auxiliar scalar field, may be implemented directly into the action, since 
the Fadeev-Popov procedure gives a constant contribution to the measure of the functional 
integral. We obtain the following Lagrangian density 



L = 2y — det [G MN + G Mp G NX *HP*y/=Go) + ~*H^d B^ + - eflupXa —*H^H x ° (1) 

Where B denotes the antisymmetric gauge field and H is the self-dual field strength H = dB. 
The 6 dimensional world volume indices are denoted by M, N = 0, 1, • ■ ■ ,5 while the spatial 
ones by fiu = l,-- - ,5. Gmn in terms of the 5-brane maps X a is given by the induced 
expression Gmn = dMX a d^fX a , where a denotes the D = 11 Minkowski indices. In the 
spatial 5-dimensional world volume we denote: 

Hpx a = d p Bxa + d a B p x + d\B ap (2) 
= \e^ sx H^ sx (3) 
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Where *H^ U is a contravariant density. In order to analyze the Hamiltonian structure of this 
action it is convenient to introduce the ADM parametrization of the metric 



Goo = -{n 2 -N x N x )=X a X a 

= N/j, = X a d fJi X a 
G^ u = 9fiv = d fJi X a d l/ X a 



G 00 
G 0fl 



GT = g" v - 



1 
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N fl N u 



n 2 ' 



9 ' 
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det(G MN ) = -n det(^). (4) 
They allow us to rewrite ([!]) in the form 



L = 2n^g~M - -^N»V p + l -*Wd B, v (5) 



where 



g = det g^ 
M = 1 + y + z 
y = l -g~ l *H^H aP 

= e^fH^IP* (6) 

The spatial world volume indices are raised and lowered with the induced metric, the need of 
using the inverse of g^ u will be later on relaxed. The term gM is precisely 



det(g^ + g- 1 / 2 *H llu )=gM (7) 
The conjugate momenta to X a may be directly evaluated. It is 



n a = 2?—(-X a + N x d x X a )M^ 2 - -V p d p X a (8) 
n 4 

We can guess that the diffeomorphisms constraints have a similar diffeomorphisms constraints 
structure as in string and membrane theory. We finally obtain 

4> = ^U 2 + g(2y + 2)=0 (9) 
4> a = Tl a d a X a + ^V a = (10) 

The conjugate momenta to B^ u will be denoted P^ u and satisfies the constraint 



or = p** - = o (ii) 

The standard canonical analysis shows that (0), (TO) and ([TTj) are the only constraints of the 
theory and they are a mixture of first and second class constraints. The canonical Hamiltonian 
is a linear combination of these constraints 



H c = A0 + A Q a + A^ 



(12) 



We will now consider a more general canonical formulation with first class constraints only, 
which under partial gauge fixing reduces to (|i~2"D. Such a formulation can be obtained by 



several approaches. Following [II], we deduce from flj~I|)the fi rs t class constraints for the 
antisymmetric field 



tt 5i = P 5i -*H 5i = (13) 
= d^ j = 0, i = 1,2,3,4. (14) 

The diffeomorphisms constraints must be modified in order to obtain a complete set of first 
class constraints and we propose the following structure: 
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l -U 2 + g{2y + 2)=Q (15) 
Tl a d a X a + \v a = (16) 



where 



V = l -g-\P^ + *H^){P aP + *H aP ) (17) 
V» = \e m ^ 5 (P aP + *2P*)(F* + W S ) (18) 

Q 5t and Q l commute between themselves and with and <p a . The simplectic space of the 
antisymmetric field and its conjugate momenta has been decomposed in terms of the local 
commuting coordinates 



p/j,v _|_ 



and 



piiv *JJIW 

with Poisson brackets 



{P"" + *H^ V , P aP - *H at3 } = (19) 

the latest coordinate may be fixed by a gauge condition associated to constraints ([13]) and (fH]). 
The procedure to obtain (|T5D, flH[), (13) and (|T^) is deductive, however we have only presented 
the resulting expressions whose required properties may be directly checked (j) and 4> a 



generates the algebra of diffeomorphism on the world volume: The canonical Hamiltonian is 
then the linear combination of the first class constraints. We have broken even further the 
manifest covariance of the formulation, nevertheless we have now a polinomic formulation of 
the theory in terms of first class constraints only. We remark that the square root characteristic 
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of the Born-Infeld action for the 5-brane has now disappear. These properties ensure the 
construction of a polinomic physical Hamiltonian and a polinomic BRST invariant formulation 
of the theory. We will consider both points in the following sections. 
The algebra generated by the first class constraints fll3|), Qli), fll5|) and (16|) is given by: 



{4> P A',} = ^d p S+ ^5 - (d^l^e^p ■ 6 (20) 
{0 P ,0'} = {(j ) + ( ) ) ')d p 5-A{d x P Xa )l ap -5 (21) 
{0,0'} = (C^^ + C'^^dJ (22) 



where 



C aX = A(gg aX + g al} l aa l px ), l afi = ^(P a/3 + *H a/3 ) (23) 

We remark that the first class constraints, as well as the structure functions of the algebra, 
depend only on g^ v and not on its inverse. This is an important difference with respect to the 
covariant formulation of the theory, where the metric is assumed to have an inverse. It is well 
known in the case of the D = 11 supermembrane that the singular configurations, which in 
this theory are string like configurations, are responsible for the continuous spectrum of the 



Hamiltonian as well as for topological changes of the brane. [ I5| , | 16|| [ |17|j . 

3 BRST effective action 

We have succeeded in finding a formulation of the 5-brane which is polinomic on the fields and 
with first class constraints only. We will now construct the associated nilpotent BRST charge 
and the BRST invariant effective action. The BRST charge is the fundamental geometrical 
object in the quantum analysis of the theory and in the construction of the map relating the 
5-brane formulation to a non commutative geometry. 

The effective action will depend on the covariant induced metric only, and not in its inverse, so 
is well behaved even on singular configurations where the determinant of the induced metric 
becomes zero. 
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In order to have a realization of the generators of spatial diffeomorphisms <p p , in the whole 
space of geometrical objects including the antisymmetric field and its conjugate momenta we 
will consider a reduction of the phase space by restricting to configurations satisfying 



d„P^ = (24) 
We may performed this restriction starting with the whole extended phase space and imposing 



finally at the level of the effective action a canonical gauge fixing condition associated to (|24]): 



X (B,P) = (25) 

depending only on B and P fields. 

Otherwise we may start restricting the phase space by conditions (24) and (25 ) and working out 



the nilpotent BRST charge on the subspace of phase space. In our case, the latest approach 
becomes slightly more direct. This is so because d^P^ commutes with all constraints of 
the theory and with all phase space coordinates except B^ u . However B^ u appears in the 
constraints only through *H al3 , which commutes with d^P^. Consequently, in the BRST 
construction we may work directly with Poisson brackets even so we are restricted by ([24]) and 
(|25|) (Dirac brackets are the same as Poisson brackets). We will follow this latest approach, 
for simplicity we require 

{X,X'} = (26) 

The algebra of diffeomorphism obtained in section is an open algebra, that is, the structure 
functions depend on the fields. Consequently, the construction of the nilpotent BRST charges 
requires several additional steps beyond the standard construction for a closed algebra [|l8] . 
They involve the higher order structure functions of the algebra. We introduce the ghosts C 
and C p associated to </> and p and its conjugate momenta /i and /i p respectively. We start 
from the Poisson brackets 

{<t>A'} = {C <TX <j><, + C'° x ct>'°)d x 5 (27) 
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where 



C x = 4(gg° x + g aP l™f x ) (28) 

It is convenient to extend the other generator of diffeomorphisms <p p from the beginning in 
order to simplify the construction. We define 



4> p = 4> p + 2fid p C + d p /i-C + fi x d p C x + d x (fi p C x ) (29) 

We then have 



i(e<P P ),c} = -2ed p c+d p (ec) = -ed P c+d P ec, (30) 

= -Z'd^-tdtfn (31) 
with the right density weights to obtain 

{{i p 4> p )^C} = -ed P (fiC) - d P Z»(fiC), (32) 

Consequently, we have 

{(^),(^} = (-OT)>=0 (33) 

We will drops all boundary terms, that is we will assume a closed compact spatial world 
volume. We also get 



{(e> P >, en = -ed P c» + d P ec p , (34) 

since C M transforms as a contravariant vector 

The introduction of <p p simplifies the construction since its action on any new term in the BRST 
charge reduces only to count weights for objects which transform under diffeomorphisms as 
densities. 
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We may start considering 



Q = (C<p + C p (f) p - fi a Cd x Cr x - C p fi x d p C x ) +Q H + Q A = Q C + Q H + Q A (35) 

Where Q H denotes the terms involving the higher order structure functions of the diffeo- 
morphism algebra. Q A denotes the contributions to Q of the constraints associated to the 
antisymmetric field. It commutes with Q c + Q H , hence we will consider it after the complete 
evaluation of Q c and Q H . We will systematically add the higher order terms in derivatives of 
C in order to ensure the nilpotency of Q. 
We get 

{(C<j>),(C<f>)}=(2Cd x CC pX <l> p ) (36) 
2{(Cj p ), {-fi a Cd x Cr x }} = -(2Cd x CC p %) - 2{[ n d p C x + d^ p C x )]Cd x CC px ) (37) 
2{(-fi a Cd x Cr x ), {-C p H d p C x )} = 2{Cd x Cr x ^ v d a C v ) - 2(Cd x CC x d p (C p ^)) (38) 

2{(Cj p ), {C x 4> x )} = -2C v d v C p $ p (39) 
2{{Cfa), (-C^ x d u C x )} = 2C v d v C P 4> p (40) 

2{{C° H doC x ), (C^ x d u C x )} = (41) 

All these brackets cancel between each other, which is the standard situation for a closed 
algebra. However, since C aX are field dependent, we have additional contributions to the 
evaluation of the {Q, Q}. 
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We obtain 



2{(C<j>), (-^Cd x CC° x )} = (2Cr aX ^Cd x Cd fl Ccf )u } (42) 

where 

c m*\ = 4(_ 2 gg^g^ + gg^g uX + - l^l vX - r CT /^ A ) (43) 

The first contribution to Q H in then 



Q H = (—CP^^CdxCd^C + ■■■) (44) 



Notice that the commutator 

{(-CO^dpCP), ~C^' aX ^ a Cd x Cd^C}, (45) 
gives a contribution which combines with (f42"D to give 

^^^CdyGd^y) (46) 
which is canceled by the commutator 



H(C p <t> P ), ~Cr aX ^ad x Cd^CC}, (47) 
Q^u,a\ - g a g a j n field dependent so there are further contributions from the commutator with 

The next contribution is from the commutators 



2{(C</>>, (--Cr aX ^d x Cd,CC)} + {{-fj, a Cd x CC° x ), (-^Cd x CC° x )} (45 
which require the addition of a term 

12 



(8g(ix x d x C) 3 C) 



(49) 



to Q. 

The commutator of (C p 4> p + /i x C p d p C x ) with © 

cancels the commutators in (|48|). Furthermore, the following commutators 



2{(C4>), (8g(fi x d x C) 3 C)} + 2{(-^Cd x CC° x ), (_Icf^ A fi uf i a Cd x Cd p C)} (50) 
require a higher order term in Q. It is 



(10g^ x d x C) 4 C) (51) 

The commutator of (C p <p p + n x C p d p C x ) with (|51~D cancels the commutators in (|50"1). 
Finally the commutators 



2{<C0>, (10^ A d A C) 4 C)} + {(--CF'^WrCdxCdnC), (--Cr aX ^Cd x Cd^C)} (52) 
require a final new terms in Q: 



{(12g^ x d x CfC)} (53) 



The commutator of (C p (j) p d p C ) with fl5"3"| ) cancels the commutators in (52). Finally the 



commutator of (C<f>) with (|53|) gives zero since the order in derivatives of C is 6. The nilpotency 
of Q has then been obtained. 

The final form of the BRST charge is thus given by 

Q = (Ct + Cfa + C^fiadxCC + /2 X C p d p C x 

- hz^ x ^ a d x c^d p cc + sg^ x d x cfc 
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+ l$g(^d x C) A C + \2g^ x d x CfC + Cdifj + Ci(P u -*H U )) (54) 
Where we have also included Q A , the contributions of the first class constraints associated to 



the antisymmetric field, which under assumption (24) become a reducible set 



We may now construct the BRST invariant effective action. We follow the BFV approach [{19 



201, and we obtain 



S eff = J d 5 adr fxC + [i p & + P^B^ + U a X a + fjCi + jld + 5(Xfi + X p fi p + X^ + Xp) 
+ S(Cx + C pX p + CXi + d 1 xi + C 2 X2 + C 3 X3 + C iX 4) (55) 

Where A, X p are the Lagrange multipliers associated to the diffeomorphisms constraints while 
Xi is associated to the constraint on the antisymmetric field. Xi X p an d Xi are the associated 
gauge fixing functions. X i-> X2, X3 an d X4 are the gauge fixing associated to the reducibility of 
the constraints on the antisymmetric field. We take the following gauge fixing functions: 



X = A- ' 



f w 

x p = y ' 

Xi = Ai (56) 

Where we have introduced a non singular metric over the spatial world volume. This is so, 
because A transforms as a density under diffeomorphisms. In some particular cases we may 
take W = 1. The introduction of this metric occurs in a similar way as for the D — 11 
supermembrane, through the gauge fixing procedure. 

Xii X2, X3 an d Xi fi x the longitudinal parts, with respect to the covariant derivative constructed 
with the metric that has been introduced, of C\ Ci, Bi and Aj, where 5Ci = Bi, 5 denotes 
the BRST transformation. We refer to for details of the construction. 



We may now integrate out, or eliminate from the field equations, the auxiliary field. We obtain 
finally: 

S eff = [ d 5 adr[fiC + fi p C p + P pv B pv + U a X a + ft i C l + JlC + -jLtf/u] (57) 
J VW 
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where 5fj, is the BRST transformed of [i. 
We notice that in the construction 



{Q,Q'}v = {Q,Q'} = 



(5? 



Where the first bracket is a Dirac bracket, while the second one is a Poisson bracket. Moreover 
the Dirac bracket of Q with any coordinate of the phase space, excluding B^ u , is the same as 
its Poisson bracket. However the Dirac bracket of Q with (P^B^) is the same as its Poisson 
bracket, in the subspace (pl[) . Consequently the effective action is BRST invariant, since its 
kinetic term transform as 



SS eff = / d 5 adTS[ijC + {i p C p + P^B^ + U a X a + fi l d + JuC] = / drQ = 



(59) 



provided initial and final conditions on the ghost fields are imposed PU| |^TJ , as usual. 
We notice that 



(-=(0 + C° x ii*d x C - -Cr aX ^d x Cfi u d,C + 8g(fi x d x C) 3 

10g(fi x d x C) A + 12g(fx x d x C) 5 )) (60) 

is not only manifestly BRST invariant, but also well behaved even on singular configurations 
of the induced metric, where its determinant is zero, in spite of the fact that fi x = g Xo 'fj, a . In 
fact all these term may be rewritten in terms of the totally antisymmetric symbol e af3fJ-uX and 
the covariant metric g pu only. 

4 Light Cone Gauge Hamiltonian of the M5-brane 

In the previous section we obtained the general formulation of the BRST invariant effective 
action of the M5-brane in the covariant gauge (|56l) , we showed that the action may be expressed 
in a manifestly Lorentz covariant way (in the target space), without restricting the induced 
metric to have an inverse. In this section we will analyze some stability properties of the 
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effective action. To do so we will start from (^) and fix the light cone gauge, which leads to 
physical Hamiltonian after elimination of the constraints. 
Consider the light cone gauge fixing conditions: 



X + = n+r (61) 
n+ = Tl+VW, (62) 

We will later on discuss the gauge fixing conditions for the antisymmetric field. 
The LCG allows to reduce canonically the phase space to its transverse part only. This is 
achieved by solving explicitly the constraint ( |T5|) for X~ is eliminated from the constraint 
fll6|) provided an integrability condition is satisfy. This condition is a first class constraint 
which generates the volume preserving diffeomorphisms. 

The canonical reduction of the effective action yields after the elimination of the ghost, 
antighost fields and Lagrange multipliers to the canonical Lagrangian: 



L = U M X M + P^B^ - H p (63) 



where 



n v = ^n M n A/ + 9 2( y + 1) + e 5l fi 5i + e^' + A a ?n af3 (64) 

M denotes the light cone transverse coordinates M = 1, ■ ■ • ,9. The explicit expression for 
y is given in (p!7[). A a ^ is the antisymmetric Lagrange multiplier associated to the volume 
preserving diffeomorphism generated by Q a /3- The explicit expression for VL a p is 



Q aP = d p {—^=[U M d a X M + \v a }} - d a {—^=[U M d p X M + \V P ]}. (65) 
IIq yW 4 IIqV^ 4 

= is the local integrability condition in order to eliminate X~ from (|i~6l). There is also 
a global integrability condition given by 



' :[U M d a X M + ]v a ]da a = (66) 



n W L 4 
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which ensures that X is a uniform scalar over the spatial world volume. C is a basis of 
homology of dimension one. If a compactified target space is assumed, the right hand side of 



(66) may be proportional to an integer. 

We have not fix the gauge associated to the antisymmetric field. There are at least three 
interesting partial gauge fixing conditions in this respect: 

• The gauge fixing yielding 



as used in the formulation in 
The gauge condition 



B 5l = 



which after double dimensional reduction of the theory yields the formulation of 4-brane 



in terms of the antisymmetric field JIT 



The gauge 



where T denotes transverse with respect to the derivatives operation. It yields after 
double dimensional reduction to the formulation of the 4-brane in terms of the Born- 
Infeld U(l) vector field |]TT|. 



The absolute minimum of the Hamiltonian is obtained at the configurations satisfying 



n a = o, 
= 0, 
i^g^gup = o, 



(67) 



over any point of the spatial world volume E 5 , which we assume is closed without boundary. 
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If = 0, then the set f2 in the space of physical configurations at which the minimum is 
obtained, becomes the set of maps X a from £5 to the target space, depending on four linear 
combinations of the local coordinates. That is, all maps X a are functions of at most four 
of them. It is an infinite dimensional space of 1,2,3 and 4 branes. The degeneracy of Q 
is analogous to the one that occurs for the D = 11 super membrane. There are string like 
spikes in that case, which are responsible together with supersymmetry for the continuous 
spectrum of the supermembrane. The degeneracy of the world volume may be pictured as 
lower p-branes emerging form the world volume which may have free ends on not. It can 
happen that the other end is plugged into another disconnected sector of the world volume. 
Such configuration is physically equivalent to the disconnected one, because the tubes not 
carry any energy. There is then, instability even in the topology of the membrane p] . These 



are general features of all p-brane which are also valid for the 5-brane, in spite of the fact that 
the known covariant formulation imposes restrictions to those singular configurations. The 
next question that one may ask is if these instabilities go away if the antisymmetric field is 
of maximum rank, using the fact that it enters in the action as a quadratic form. That is, if 
we consider the antisymmetric field living on a nontrivial higher order bundle (thus avoiding 
the = configuration), can we still have degenerate configurations?. The answer to this 
question is that even if we assume the antisymmetric field of maximum rank at any point of £5, 
there may be string like spikes with zero energy emerging from the world volume. This is so 
because the antisymmetric field at any point of £5, has always at least one zero eigenvalue. 
Its eigenvector being the topological vector V^, which is independent of the induced metric. 
In fact 



rV v = (68) 

at any point of £5. 

If is a zero vector, then has three eigenvectors with zero eigenvalue, and the following 
argument will be also valid. 

We consider the following string like configurations 



X a = X a (Y), 
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d,Y = <f>V^ (69) 
where <fi is scalar field over S 5 which will be determined. 

We may take the family of curves over S 5 tangent, at any point of an open neighborhood, 
to Vp. We assume ^ on that open set. We then choose the cr 5 coordinate along these 
curves. We then have: 



d t Y = 0, 

d 5 Y = <j)V 5 , (70) 



The solution to this equation is given by 



, d 5 Y = f(a b ) (71) 



V 5 

Where / is an arbitrary scalar field over E 5 . 

We then conclude, that the quadratic term in the Hamiltonian arising from the antisymmetric 
field is zero for any of these string like configurations. That is, even when is of maximum 
rank at any point of S 5 the world volume can degenerate to acquire string like spikes with 
zero energy. We notice that when IT a = 0, as we are assuming, the admissible configurations 
for the antisymmetric field are restricted by 

d,X~ = ~V„ (72) 

even so there are admissible configurations with ^ and of course with = 0. In the 
latest case we replace in the above argument by one of the eigenvectors of with zero 
eigenvalue. 

Finally, we discuss topological conditions which prevent (classically) the existence of degen- 
erated spikes in the world volume. The condition we are going to discuss is not related to a 
BPS bound in the case of the 5-brane. 

The integral of the determinant of the induced metric may be expressed as: 



It, 5 



{dX a A dX b A dX c A dX d A dX e ) * ( ^ Xa ^'^ dXe ^j (73) 
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Where * denotes the Hodge dual. A nontrivial minimum of this expression is achieved when 



*U = * (dX a A • • • A dX b ) = constant • integer (74) 

for a set of five maps, X 1 , • ■ • , X 5 . U is then the curvature of a potential 4-form living on 
a non trivial higher order bundle |13 |, [p2 |. In particular (|73|) implies that dX 1 , • • • , dX 5 are 
closed, non exact, 1-forms. 

We may now show that the set Q of configurations at which this local minimum is obtained is 
finite dimensional. We consider any other set of maps X 1 , ■ ■ ■ ,X 5 on the same higher order 
bundle. We have 



dX 1 A ■ • • A dX 5 = U + 5U (75) 
where 5U is exact. We then obtain 



/ (U + 5U) * (U + 5U) = f U*U+ I 5U*SU> f U*U (76) 

The equality is obtained when 5U = 0. The question is then if we can perform a deformation 
of X 1 , • • • , X 5 such that U is preserved. This is always possible since we may consider 



5X 5 = f(X\--- ,X*) (77) 

where / is an arbitrary scalar over £5. It seems then that the same infinite dimensional space 
of configurations Q will still exists. However, under assumption (|74[) , they can be removed by 
a volume preserving diffeomorphism. In fact, if U is preserved under the deformation, SU = 0, 
then the deformed potential must satisfy 



U 
5V 



dV 
dA 
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(78 



However, under volume preserving diffeomorphisms with group parameter 



e = e^d v X Xpa 



(79) 



we obtain 



5 (x a dX b A dX c A dX d A dX e e abcde ^ = d{*UX), (80) 

*U being constant by (0). We may then cancel the deformation and remove the degeneracy 
of Q arising from exact forms in ([7£|). We are thus left with the cohomology classes of closed 
forms only. The functional space Q of minimal configurations is then finite dimensional. 

5 Algebra of diffeomorphisms on a 6 dimensional world 
volume 

We will find in this section all possible field theories, realized in terms of the fields X a and 
Bfj, u which are invariant under 6 dimensional diffeomorphisms. To do so, we will find the 
general structure of the Hamiltonian constraint by imposing the closure of the algebra of 
diffeomorphism. It turns out that the only polinomic constraint which depends on g pv and 
not on its inverse is the associated to the 5-brane theory. The algebra is the following 



{0^} = ^d p 5 + ^8,5 - (d^l^e.spap ■ S (81) 
{0 P ,0'} = {(f ) + ct ) ')d p 5-A{d x P Xa )D ap -5 (82) 
{0,0'} = (V^ + C^^dJ (83) 

where the prime denotes evaluation at a point on the world volume of local coordinates 
{&[,■ ■ ■ , 05). C pa and D ap are local functions of the canonical variables. (fi p may be real- 
ized in terms of the "topological" expression ( |T6| ) (since it is independent of any metric on the 
world volume) 
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54) 



where 



jpfw _|_ 



(86) 



We propose for 4> the expression 



-irn a + w 



= gF(y,z) where F is to be determined in order to satisfy (|83| ). We notice that (|82 



57) 



is 



satisfied by any scalar density of weight 1, in the sense 4> = g ■ (scalar field), since 4> p given by 
(|84]) generates the diffeomorphisms on the spatial 5 dimensional sector of the world volume. 
The general solution for F(y, z) is given by the space of solutions of the partial differential 
equation -We refer to Appendix A for the detail calculations of (|83|)-. 



dF fdF\ 2 OFdF f dF\ 2 „ dF OF „ , 

2 7r z+ (ir) z + yir7r + (ir) =2F ~ 2z 7r -^tt' 

oz \oz J oz oy \0y J oz ay 

In particular, the following is a solution of (551): 



F = 2y + X^l + y + z + ^ + 2 

for any value of A. 

The final form of the constraint for this solution is then given by 



39) 



^ a n a + g 



2y + X^/l + y + z + — + 2 

o 

and replacing the expression for y and z we have 



A 2 



(90) 



2n a n a + s 



g-HapV* + + \g-n^ + Lg-igv>VrV u + y + 2 



(91) 
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When A = we obtain the constraint associated to the Lagrangian ([!]), which is quadratic on 
the field B^. It is interesting to study which could be the Lagrangian associated to the more 
general constraint (91). It turns out to be the Lagrangian ([!]) with cosmological term j\fG 
added. 

Indeed, the canonical analysis of such an action along the lines presented in section (|2|) yield 
constraint (|9l|) . Explicitly we have 



L = 2n v^M - ~N p V p + ~*H» u d B^ + ^ng 1 ' 2 (92) 



from which we obtain 



IT a = 2^(-X a + N x d x X a )T - jV p d p X a (93) 



where 



T = (M 1/2 + -) (94) 

and from (^) -instead of (§)- we obtain flg|) and (0) in the gauge = . 
We notice in ©, that the term gg^ = ^ VlV2U ' AVi g UlXl g U2 x 2 g U3Xz g UA x A e uXlX2XzXA , hence it may 
be rewritten in terms of the covariant metric g^ v . The expression (|9~l"l) , consequently, may be 
expressed in terms of g^, it is not assumed the existence of the inverse. In order that the 
scalar z may be involved in an expression with that property, it must appear as 



z a , a< 1 - (95) 



The expression will then be non-polinomic. The most general polinomic solution to ( p8|) which 
does not depend on the inverse metric is then 



F = ay + \ 2 (96) 

for any real number a ^ 0. However a must be at least a > in order to have a bounded from 
below Hamiltonian. It then may be rescaled to a fix number since the transformations 
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n — ► Att i — >i, 

X — ► \X a — > A 6 a, (97) 
<P — > A 2 . 

leave the canonical Lagrangian invariant. The property, of the realization of the algebra, for 
the M5-brane of being only dependent on g^ u and not on its inverse is preserved when we 
consider a dimensional reduction to 5 dimensional world volume. If we impose the double 
dimensional reduction, in the sense 



x 5 = a 5 



^ = < 9f 



we then obtain the following realization of the algebra 



= fydiS + tifyS - (d k P kl YH 5m e ljlm 5 (99) 

= {(j ) + <i ) ')d l 5-A{d l P lm )l mi -5 (100) 

{0,0'} = {C i % + C Hk <l>W (101) 

where, 



fc = U a d k X a + l -e mnlk P mn *H l (102) 
4> = ^n 2 + 2g + 2(^P kl g ik g, l + ^H i ^g lJ ) + ^e mnlk P mn P lk ) 2 (103) 
C ik = A(gg ik + -P ij P kl g jl + *H i *H k ) (104) 

*H* = ^ kl H jkl (105) 

This theory incorporates the singular configurations of the metric to the 4-brane theory. It 
does correspond to it in the flat limit [II| and couples consistently the antisymmetric field 
with the induced metric. However a direct check showing the equivalence of the canonical 
Lagrangian, arising from these constraints, and the usual 4-brane one has not be performed. 
The Hamiltonian constraint is now quartic in the antisymmetric field. We have explicitly 
checked the closure of this algebra with the structure functions given. It exactly correspond 
to the dimensional reduction of the 6 dimensional algebra previously obtained. 
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6 The subspace of solutions with flat induced metric 



In this section we shall show how can we recover the master canonical action lITlfl — with 



the flat induced metric — starting from the canonical action constructed with the canonical 
Hamiltonian (|l|), which is quadratic in the antisymmetric field. To achieve this goal, we will 
rely on a particular solution of the field equations 
Lets consider the field equations arising from the canonical Lagrangian 



L = U a X a + P^B, W - H (106) 

where 



n = a<p + A a (f) a + e 5i fi 5i + (107) 

is the canonical Hamiltonian with the general form of the Hamiltonian constraint obtained in 
section (|5|). 

By taking variations of ( |106j ) with respect to n a and X a , A and A a we obtain 



0=0 (108) 

cj) a = (109) 

X a = AU a + A a d a X a (110) 

-Ha = A^-d a [A a Tl a } (111) 

We now consider the subspace of solutions of the field equations which satisfy initially 



X° = T 

X a = a a a= I,-- - ,5 

X a = 0, a > 6 

n a = 0, a>6 (112) 



We impose gauge conditions in the Lagrange multipliers 
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A a = -AIF, (113) 



We then obtain from (|108|) , (|109|) , ( |110| ) that ( |112|) are preserved by the evolution equations 



and X° = r. The explicit expressions for the constraints ( |108|) and (|109|) , ( |91~D and (|T^) 
respectively, provide us the values of n and IT Q in terms of the antisymmetric field and their 
conjugate momenta, specifically from ( |109| ) we can get the values for U a 



n Q = --v a (ii4) 



and replacing this result in ( |108| ) we obtain II 



ro 



+ ±V a V a + g(2y + ^ + AM 1 / 2 + 2) = (115) 



II = 2 (V/ 2 + ^ (116) 

where M = 1 + y + z. 

Replacing the expressions for n and IIq, in the canonical Lagrangian ( |106| ) we finally obtain 
the canonical Lagrangian over a flat induced metric, 



L = P^B^ - 2 (V 1 / 2 + @5^ 5i + ®jW (117) 



When A = we recover the master canonical action |TTJ for the Perry and Schwarz formulation 
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7 Conclusions 

We performed a complete canonical analysis of the bosonic M-theory five brane action cor- 
responding to the partial gauge fixed formulation of the PST action where the scalar field is 
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fixed as the world volume time. This canonical formulation is quadratic in the dependence 
on the antisymmetric field and it has second class constraints. We removed the second class 
constraints by proposing an extension of the canonical action derived from the covariant action 
of m and constructed a master canonical action with first class constraints only, preserving 
the locality of the field theory. We then constructed the associated nilpotent BRST charge, 
assuming a world volume with a compact without boundary spatial part, and its BRST in- 
variant effective theory. The BRST charge is well defined even for configurations in which the 
induced metric has zero determinant at some point or open neighborhood of the world volume. 
It does not require the existence of the inverse of the induced metric. Consequently, the BRST 
effective action, which is manifestly Lorentz invariant in the target space and manifestly BRST 
invariant in the world volume, has also the same property. The singular configurations have 
then to be considered as physical one. This implies the existence of configurations changing 
the topology of the M5-brane without changing its energy. We expect then that they will have 
the same consequences as for the D — 11 supermembrane with respect to its spectrum once 
the supersymmetry is implemented into the theory. 

We obtained the physical Hamiltonian of the theory in the LCG from the general effective 
action (|55|) and analyzed its stability properties explicitly. We showed the existence of singular 
configurations, where g = 0, even for maximum rank of the antisymmetric field. We also 
constructed global configurations where singularities are not allowed at any point of the world 
volume, and give a geometrical interpretation of them in terms of higher order bundles. 
Finally, by studying the algebra of 6 dimensional diffeomorphisms we found the most general 
structure for the Hamiltonian constraint and we identified the constraint associated with the 
bosonic five brane action upgraded with a cosmological term as a constraint with a Born-Infeld 
type term. The M5-brane may be characterized from this algebraic point of view as been the 
only one whose Hamiltonian constraint is polinomial in the antisymmetric field and is well 
defined without assuming the existence of the inverse of the induced metric. 
We have now all elements to start analyzing the spectrum of the super M5-brane. In particular 
to look at the massless states of the theory. It will be important to relate this problem to its 
dual one for the D = 11 super membrane, where the spectrum was determined only for flat 
Minkowski target space using an SU (N) regularization. Another aspect that may follow from 
our results is the explicit construction of the Seiberg-Witten map relating the M5-brane to a 
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non-commutative geometries in terms of the BRST cohomology. In this sense, the simplectic 
structure we have constructed may be important. 
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A Appendix 



We will evaluate (jig) in terms of the functions F(y, z). 
We have 



{0, <P'} = {^ttX + gF, IttX + 



(118) 



which by explicit evaluation yields 



{0,0'} 



7T 



K, g'}F' + </-^K, z'} + 2/'} 



i r n / if 'i / i r 'i / 



<9F r n ,<9F' 



(119) 



The Piosson brackets of P^ u and *H ^ is given by 



2 L ' P Acr J 

-& p to v &Mo' - a) 



(120) 



we then obtain 



2 p 



(121) 



We also get 



^'[{-^'HW'}] 



1 



-64' 



32 
1 

32 



32 



(122) 
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and 



gg'{z, z'} 
{KX} 

gg'{z, z'} 



64 64 \ v >" v »t 

1 1 1 r 
41616 



2 

TV icr\ 



(123) 



We finally obtain 



{W, W'} = I + II + III 



(124) 



where 



dy dy 4 



d x 5 



II = 



dxS 



III = 



5a5 



(125) 



We will now evaluated Poisson brackets related to X a and Il a . 
We get 



-2&J{a' - a)%x' a 



(126) 



which may be used to evaluate 



-{n a n a ,W>} = n a [{7r a ,g'}F' + {ir a: F'}g'} 



= 7T 



OF' dF' 

K, g '}F' + g'-^Ua, z'} + g'-^Ua, y'} 



(127) 



we obtain after some calculations 
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-{n a n a ,W'}=A + B + C 



:i28) 



with 



A = [2Fgg°\ a d a x a + f ] d x 5 



B 



C 



3P arp 



1 
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BF OF 

^yWa ff i a + g^ a g V l j,V v 7r a d rT x a + / 



<9 A 5 (129) 



We now use ( |125| ) y (|129|) to obtain the closure of the algebra of constraints, that is to 
restrict F(y, z) in order to satisfy ([15]). After some calculations we obtain the following partial 
differential equation to be satisfied by F, 



2 <9F /dF\ 2 dFdF /<9F X 2 
dz Z \ dz J ^ dz dy \dy 



dF OF 
2F - 2z— - 2y— 

oz ay 



(130) 
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